ci: Port lint scripts from bitcoin #1823
Conversation
div72
changed the title
|
Oof... |
|
Nice start. Some good stuff in there. A lot of noise, though. Will take some time to knock out the whole rule set. We may plug this in with |
|
This is really useful and pretty much exactly what I was trying to get working, just in a different way. Thanks @div72 ! |
|
A bug with lint. It is treating an escaped % in a LogPrint argument as an argument format specifier. |
|
Our position on boost is different than Bitcoin, so we need to adjust those rules. |
|
All calls to LogPrintf() and LogPrint() should be terminated with \n. This is not true with us, and we should change the rule. This is a stupid rule and when I ported the logging system I changed the underlying functions to put the \n terminator in there underneath so I did not literally have to go through every single LogPrint statement and change each one. Give me an example (realistically) of where you want to do a logging statement and it shouldn't have a \n termination. Exactly. There isn't a realistic one. |
|
A general comment about Bitcoin... those people are not God, and just because they do stuff doesn't mean we will automatically follow. We have to keep in mind the rebase, but we will NOT be slaves to them. |
Agreed, although I won't complain if we decide to follow their decision about boost. |
|
Also doesn't |
|
On Boost... I have mixed opinions. Many things that have ended up in the std libraries originated in Boost (with slight changes of course). Be that as it may, if standard library implementations suffice, we should use those. One of the biggest gaps is the boost filesystem classes, which are really elegant. IIRC those are being put in std (with some tweaks as usual). We are also using some of the boost extended stream implementations, including filters, and some of the calendaring classes which are also very useful. My view on this is that we should minimize the use of boost to only those areas where there is not an acceptable std library implementation... for example, we ought to be able to rip out all of the boost thread handling and replace with std implementations. |
|
Bitcoin uses Boost (and other third-party libraries like OpenSSL) conservatively not as a form of elitism. The project suffered several consensus failures in the past because of subtle behavior variations between library versions and their implementations for different platforms. For this reason, the maintainers review the use of third-party libraries very closely. Gridcoin experienced forking issues caused by Boost in the past as well. As we know, lints are a sanity check tool, not an injunction. They draw human attention to code that may need closer inspection. The Boost rules do not ban the use of the library—these ensure that no one overlooks the use of those libraries in consensus-critical code. I continue to support the application of Boost where it makes sense. We should minimize its presence where we have better tools. |
|
I agree, and I am not arguing that the Bitcoin dev's attitude towards third party libraries is based on elitism... The snottiness there extends across the board I think... but they do have some talented folks working on Core, and they are justified in their attitude given the status of Bitcoin Core in the world of cryptocurrency. We are in agreement. In fact I would argue we should minimize its presence if we have adequate tools in the standard libraries, and should only use Boost or other third party libraries if their use prevents onerous work, or the risk would actually INCREASE if we did NOT use them. That could be true for specialty libraries where a custom coded replacement could actually have more problems than the library you are trying to replace! On the use of other third party libraries, I agree we need to control the dependencies, especially in consensus-critical code. For example we should finish the port of Bitcoin's replacement of OpenSSL for the purposes of signatures, etc, although we will have to maintain the OpenSSL dependency for the sole purpose of CURL. (I for one do NOT want to try and custom code a replacement for CURL. It is very capable and we are not likely to do better...) |
@div72 Yes, it effectively does the same thing. Let's leave Bitcoin's include guards alone for now. We can update the linter to allow our use of |
|
@div72 Before you get too far down the rabbit hole, let's freeze the de-linting and review this PR as-is. We can add the lint stage to CI without blocking other PRs until we address the remaining violations (and some of these will go away as we port Bitcoin's newer code). Since many of these changes will touch a lot of files, merging a huge PR may create a lot of conflicts for other people's branches. On the flip side, continuously rebasing this branch on top of other incoming PRs will make it very hard to review. I look over non-trivial PRs commit-by-commit, and I have to start over for a rebase, so it will save me a lot of work 🙂 |
|
@cyrossignol Mind if I pull c2ed11e back? |
|
@div72 No, please do. We can reapply when the commit is ready. |
Most Gridcoin-specific headers use "#pragma once" as an include guard. This changes the lint rules to allow "#pragma once" in addition to the standard "#ifndef"/"#define" macro guards.
This allows the lint job to fail without invalidating a build so that we can address unfinished lint violations incrementally.
|
No idea why fast_finish doesn't work, but since the lint stage takes like a minute I don't think it's much of an issue. |
|
I really like this PR, but I want to discuss merge strategy a little more. @cyrossignol is right, this could cause a lot of rebase work for branches that people may have in progress, although we are at a real low point on PR's, which is a good thing in respect to this one. Also, the development branch is temporarily in a frozen state except for fixes that I deem worthy for admission into Fern, but any fixes for Fern are pointwise, and can start on a commit base post this PR merge, so I don't see it as a problem. @cyrossignol what is your opinion on this? Wait until after Fern release when the development branch is unfrozen? |
|
Yeah, but I don't want you to have to continually rebase, either. It would seem to me that the right time to do this is right after the release as the first post release merged PR... |
Co-authored-by: Cy Rossignol <[email protected]>
|
Merging. |
Added - Backport newer uint256 types from Bitcoin #1570 (@cyrossignol) - Implement project level rain for rainbymagnitude #1580 (@jamescowens) - Upgrade utilities (Update checker and snapshot downloader/application) #1576 (@iFoggz) - Provide fees collected in the block by the miner #1601 (@iFoggz) - Add support for generating legacy superblocks from scraper stats #1603 (@cyrossignol) - Port of the Bitcoin Logger to Gridcoin #1600 (@jamescowens) - Implement zapwallettxes #1605 (@jamescowens) - Implements a global event filter to suppress help question mark #1609 (@jamescowens) - Add next target difficulty to RPC output #1615 (@cyrossignol) - Add caching for block hashes to CBlock #1624 (@cyrossignol) - Make toolbars and tray icon red for testnet #1637 (@jamescowens) - Add an rpc call convergencereport #1643 (@jamescowens) - Implement newline filter on config file read in #1645 (@jamescowens) - Implement beacon status icon/button #1646 (@jamescowens) - Add gridcointestnet.png #1649 (@caraka) - Add precision to support magnitudes less than 1 #1651 (@cyrossignol) - Replace research accrual calculations with superblock snapshots #1657 (@cyrossignol) - Publish example gridcoinresearch.conf as a md document to the doc directory #1662 (@jamescowens) - Add options checkbox to disable transaction notifications #1666 (@jamescowens) - Add support for self-service beacon deletion #1695 (@cyrossignol) - Add support for type-specific contract fee amounts #1698 (@cyrossignol) - Add verifiedbeaconreport and pendingbeaconreport #1696 (@jamescowens) - Add preliminary testing option for block v11 height on testnet #1706 (@cyrossignol) - Add verified beacons manifest part to superblock validator #1711 (@cyrossignol) - Implement beacon, vote, and superblock display categories/icons in UI transaction model #1717 (@jamescowens) - neuralnet: Add integrity checking to researcher accrual snapshot registry #1727 (@jamescowens) - Add workaround for scrypt assembly on macOS #1740 (@cyrossignol) - gui: Build onboarding/beacon wizard #1739 (@cyrossignol) - doc: Add CONTRIBUTING.md from bitcoin #1723 (@div72) - rpc: Implement inspectaccrualsnapshot and parseaccrualsnapshotfile #1744 (@jamescowens) - scraper: Add disk based state backing for verified beacon list in scraper #1751 (@jamescowens) - Add ability to recover beacon in block version 11+ #1768 (@cyrossignol) - refactor: Add transaction context to contract handlers #1777 (@cyrossignol) - gui: Add context for when BOINC is attached to a pool #1775 (@cyrossignol) - doc: Clarify what to do if PR in multiple categories (for CONTRIBUTING.md) #1798 (@RoboticMind) - qt: Add option to choose not to start the wallet minimized #1804 (@jamescowens) - superblock: Add check for OutOfSyncByAge to SuperblockValidator::Validate #1806 (@jamescowens) - contract: Standardize contract validation and add block context #1808 (@cyrossignol) - add seed.gridcoin.pl to default config #1812 (@wilkart) - gui: Implement sidestake send display #1813 (@jamescowens) - gui: Add pool/investor pages to researcher wizard #1819 (@cyrossignol) - ci: Port lint scripts from Bitcoin #1823 (@div72) - doc: Create basic readme in contrib #1826 (@RoboticMind) - gui: Implement TransactionRecord::Message #1829 (@jamescowens) - rpc: Add private_key_available to beaconstatus #1833 (@a123b) - gui: Validate email address in researcher wizard #1840 (@a123b) - rpc: Add "getrawwallettransaction" RPC function #1842 (@cyrossignol) - consensus: Set block version 11 threshold height for mainnet #1862 (@cyrossignol) Changed - Upgrade LevelDB from v1.17 to v1.20 #1562 (@cyrossignol) - Re-enable scrypt optimizations #1450 (@denravonska) - Derive CScript from prevector type (optimization) #1554 (@cyrossignol) - Disable quorum for grandfathered blocks to speed up sync #1568 (@cyrossignol) - Refactor hashBoinc for binary claim contexts #1558 (@cyrossignol) - integrated_scraper_2 branch tracking PR #1559 (@jamescowens) - Upgrade depends - OpenSSL to 1.1.1d #1581 (@jamescowens) - Ubuntu 19.10 fixes #1590 (@denravonska) - Force a re-parse of legacy claims in generated blocks #1592 (@cyrossignol) - Improve the "versionreport" RPC output #1595 (@cyrossignol) - Overhaul the core tally and accrual system #1583 (@cyrossignol) - Overhaul the superblock quorum system #1597 (@cyrossignol) - Add more data to the "superblocks" RPC output #1599 (@cyrossignol) - Update Windows Build doc #1606 (@barton2526) - Change the order of calls in gridcoinresearchd.cpp to optimize rpc shunt path #1610 (@jamescowens) - Change staking tooltip to display frequency #1611 (@jamescowens) - Enhancements to ETTS #1442 (@jamescowens) - Standardize money values as integers #1614 (@cyrossignol) - Clean up and optimize legacy coin age code #1616 (@cyrossignol) - Some scraper cleanups #1620 (@jamescowens) - Reorganize accrual code and fix 6-month cutoff #1630 (@cyrossignol) - Update Copyright years #1633 (@barton2526) - Change team whitelist delimiter to <> for CPID detection #1634 (@cyrossignol) - Change team whitelist separator to <> to accomodate more team names #1632 (@jamescowens) - Change Curl download speed type to support older environments #1640 (@cyrossignol) - Optimize logo SVGs used for tray icons #1638 (@cyrossignol) - Tweak consolidateunspent rpc function #1644 (@jamescowens) - ETTS and staking icon enhancements #1650 (@jamescowens) - Implement new transaction fees for block version 11 #1652 (@jamescowens) - Optimize in-memory storage of superblock data #1653 (@cyrossignol) - Miscellaneous superblock API improvements and housekeeping #1654 (@cyrossignol) - Update openssl to 1.1.1f compatibility #1660 (@jamescowens) - Optimize bdb to avoid synchronous flush of database #1659 (@jamescowens) - Add support for CPID input to "lifetime" RPC function #1668 (@cyrossignol) - Overhaul the contract handling system #1669 (@cyrossignol) - Make the autostart mainnet/testnet aware #1671 (@jamescowens) - Remove slashes from User Agent in peers tab #1674 (@div72) - Refactor contracts for polymorphic binary payloads #1676 (@cyrossignol) - Overhaul the beacon system #1678 (@cyrossignol) - Replace boost::optional<T&> with non-owning pointers #1680 (@cyrossignol) - Optimize proof-of-stake validation #1681 (@cyrossignol) - Updated Slack link #1683 (@NeuralMiner) - Update build-unix.md #1686 (@Quezacoatl1) - Replace deprecated QT methods #1693 (@Pythonix) - Made protocol.h more similar to bitcoin #1688 (@Pythonix) - Touch up some details for block version 11 #1697 (@cyrossignol) - More tweaks for block version 11 #1700 (@cyrossignol) - Finish the conversion to the BCLog class based logger #1699 (@jamescowens) - Move claim version transitional code in miner for proper signature #1712 (@cyrossignol) - doc: Update threads in coding.txt #1730 (@div72) - qt: Include QPainterPath in trafficgraphwidget.cpp #1733 (@div72) - doc: Update doc/build-unix.md #1731 (@div72) - gui: Show peers tab on connections icon click #1734 (@div72) - refactor: Change return type of IsMine to isminetype && move wallet files to wallet directory #1722 (@div72) - build: Updates boost to 1.73.0 for depends #1673 (@jamescowens) - doc: Update Unit Test Readme #1743 (@RoboticMind) - wallet: Change Assert To Error Message In kernel.cpp #1748 (@RoboticMind) - scraper: Shorten display representation of verification codes #1754 (@cyrossignol) - log: Change ".B." to Clear Message #1758 (@RoboticMind) - util: Fix braindamage in GetDefaultDataDir() #1737 (@jamescowens) - scraper: Improve scraper processing of beacon verifications #1760 (@jamescowens) - scraper: Add instrumentation to convergencereport #1763 (@jamescowens) - rpc: Improve rpc stress test script #1767 (@tunisiano187) - Generalize enum serialization #1770 (@cyrossignol) - scraper: Improve handling of ETags in http class and tweak verified beacon logic #1776 (@jamescowens) - scraper: Improve ProcessNetworkWideFromProjectStats and other tweaks #1778 (@jamescowens) - researcher: Automate beacon advertisement for renewals only #1781 (@cyrossignol) - gui: Tweak behavior of beacon page in researcher wizard #1784 (@cyrossignol) - Prepare for block version 11 hard-fork on testnet #1787 (@cyrossignol) - scraper: Modify UpdateVerifiedBeaconsFromConsensus #1791 (@jamescowens) - gui: Optimize OverviewPage::updateTransactions() #1794 (@jamescowens) - ci: Adopt ci changes from Bitcoin #1795 (@div72) - consensus: switch snapshot accrual calculation to integer arithmetic #1799 (@cyrossignol) - voting: Overhaul the voting system #1809 (@cyrossignol) - contract: Optimize contract replay after chain reorganization #1815 (@cyrossignol) - contract: Reimplement transaction messages as contracts #1816 (@cyrossignol) - staking: Sign claim contracts with coinstake transaction #1817 (@cyrossignol) - gui: Change research wizard text #1820 (@div72) - net: Update protocol version and clean up net messaging #1824 (@cyrossignol) - rpc, wallet: Corrections to GetAmounts #1825 (@jamescowens) - gui: Tweak some minor researcher wizard details #1830 (@cyrossignol) - gui: Change GetEstimatedStakingFrequency text #1836 (@jamescowens) - scraper: Scraper global statistics cache optimization #1837 (@jamescowens) - doc: Update Vulnerability Response Process #1843 (@RoboticMind) - scraper: Optimization of manifest and parts sharing between ConvergedScraperStatsCache, mapManifest, and mapParts #1851 (@jamescowens) - consensus: Update Checkpoints #1855 (@barton2526) - docs: Update docs to build off master #1856 (@barton2526) - gui: Fix and improve GUI combo box styles #1858 (@cyrossignol) - build: Tweak Gridcoin installer for Fern release #1863 (@jamescowens) Removed - Remove old research age checks (rebase #1365) #1572 (@cyrossignol) - Remove PrimaryCPID check from diagnostics dialog #1586 (@cyrossignol) - Remove missed label for PrimaryCPID from diagnostics #1588 (@cyrossignol) - Remove legacy quorum messaging system (@neural network) #1589 (@cyrossignol) - Remove old remnants of legacy smart contract experiments #1594 (@cyrossignol) - Remove block nonce for version 11 #1622 (@cyrossignol) - Delete obsolete contrib/Installer and Upgrader directories #1623 (@jamescowens) - Remove redundant LoadAdminMessages() calls #1625 (@cyrossignol) - Remove some legacy informational RPC commands #1658 (@cyrossignol) - Remove informational magnitude field from binary claims #1661 (@cyrossignol) - Remove fDebug3,4, and net and convert to BCLog::LogFlags #1663 (@jamescowens) - Remove qt5.7.1 depends support build System #1665 (@iFoggz) - Remove unused jQuery library #1679 (@cyrossignol) - Remove unused NetworkTimer() function and global state #1701 (@cyrossignol) - Refactor claim context objects into contracts #1704 (@cyrossignol) - Clean old assets up #1718 (@div72) - Remove legacy "rain" RPC (not by-project rain) #1742 (@cyrossignol) - Temporarily disable voting system on testnet #1769 (@cyrossignol) - gui: Remove legacy GUI transaction description for contracts #1772 (@cyrossignol) - gui: Remove transaction fee setting #1780 (@cyrossignol) - trivial: Cleanup unused legacy functions #1793 (@cyrossignol) - mining, rpc: Remove kernel-diff-best and kernel-diff-sum #1796 (@jamescowens) - refactor: Remove libs subdirectory #1802 (@div72) - scraper: cleanup unused/unnecessary functions #1803 (@jamescowens) - gui: Remove useless "Detach databases at shutdown" #1810 (@jamescowens) - test: Remove testnet condition for standard transactions #1814 (@cyrossignol) - consensus: Remove transitional testnet code #1854 (@cyrossignol) Fixed - Fix "Owed" amount in output of "magnitude" RPC method #1569 (@cyrossignol) - Add support for paths with special characters on Windows #1571 (@cyrossignol) - Fix lingering peers.dat temp files and clean up remaining paths #1582 (@cyrossignol) - Fix incorrect beacon length warning in GUI transaction list #1585 (@cyrossignol) - Fix default config file line endings on Windows #1587 (@cyrossignol) - Reenable Travis builds for MacOS #1591 (@jamescowens) - Correct peer detail info background color #1593 (@jamescowens) - Fix exception in debug3 mode #1598 (@cyrossignol) - Fix deadlock in "getmininginfo" RPC function #1596 (@cyrossignol) - Fix accuracy of statistics in "network" RPC output #1602 (@cyrossignol) - Fix heights for quorum vote weight calculations #1604 (@cyrossignol) - Fix deadlock in log archiver when rename fails #1607 (@cyrossignol) - Fix a spurious segmentation fault during client load on Windows with fast CPUs #1608 (@jamescowens) - Fix lock order debugging and potential deadlocks #1612 (@jamescowens) - Add dependencies #1613 (@Scalextrix) - Fix std namespace pollution #1617 (@denravonska) - Add missing condition for newbie accrual computer #1618 (@cyrossignol) - Track first reward blocks in research accounts #1619 (@cyrossignol) - Fix lingering beacon warning after advertisement #1627 (@cyrossignol) - Fix accrual calculation for new, zero-magnitude CPIDs #1636 (@cyrossignol) - Fix diagnostics, add ETTS test, fix tooltipcolor, add missing lock, and add email=investor check #1647 (@jamescowens) - Fix help message of two RPC methods #1656 (@div72) - Fix legacy accrual for newbie with non-zero past reward #1667 (@cyrossignol) - Fix GUI autostart on Windows for paths with wide characters #1670 (@cyrossignol) - Qualify boost bind placeholders with their full namespace #1672 (@Ponce) - Fix suffix when copying txids #1677 (@div72) - Unnecessary if-statement removed #1685 (@Pythonix) - Fix consolidatemsunspent Help Message #1687 (@Pythonix) - Fix gettransaction help message #1691 (@Pythonix) - Fix GetNewMint To Look for Stakes #1692 (@RoboticMind) - Suppress deprecated copy warnings for Qt with GCC 9+ #1702 (@cyrossignol) - Fix exclusion error on stats processing and misplaced ENDLOCK logging entry #1710 (@jamescowens) - Removed unnecessary comparison #1708 (@Pythonix) - Fixed typo #1707 (@Pythonix) - Fix out-of-bounds exception for peers tab version slashes #1713 (@cyrossignol) - Fix transition for v1 superblocks when reorganizing #1714 (@cyrossignol) - Touch up transition to version 2 transactions #1715 (@cyrossignol) - Avoid mutating transactions in ConnectBlock() #1716 (@cyrossignol) - Skip beacon advertisement when already pending #1726 (@cyrossignol) - Fix Windows cross-compilation in newer environments #1728 (@cyrossignol) - Fix out-of-bounds access in IsMineInner() #1736 (@cyrossignol) - Fix a couple of block version 11 issues #1738 (@cyrossignol) - Fix null pointer dereference in GUI researcher model #1741 (@cyrossignol) - accrual: Reset research accounts when rebuilding accrual snapshots #1745 (@cyrossignol) - scraper: Correct update for verified beacons #1747 (@jamescowens) - accrual: Refactor tally initialization for snapshot rebuild #1749 (@cyrossignol) - rpc: Fix "cpid" field in "beaconconvergence" RPC output #1750 (@cyrossignol) - accrual: Fix snapshot accrual superblock state transitions #1752 (@cyrossignol) - scraper: Correct stale verified beacon logic #1753 (@jamescowens) - rpc: Correct possible divide by zero in getblockstats #1755 (@jamescowens) - gui: Fix issues with researcher wizard flow #1756 (@cyrossignol) - wallet: Stop Error When Starting From Zero #1759 (@RoboticMind) - Don't count empty email as explicit investor #1761 (@cyrossignol) - accrual: Fix snapshot accrual superblock state transitions #1764 (@cyrossignol) - rpc: Cleanup Help Message and Fix Typo #1771 (@RoboticMind) - scraper: Fix scraper etag header case sensitivity #1773 (@cyrossignol) - consensus: Use explicit time to check if superblock needed #1774 (@cyrossignol) - gui: Fix scroll area dark theme styles #1785 (@cyrossignol) - rpc, gui: Fix three divide by zero possibilities #1789 (@jamescowens) - rpc: Fix balance pre-check in "rainbymagnitude" RPC #1792 (@cyrossignol) - accrual: Fix outdated comment and correct grammar #1800 (@RoboticMind) - gui: Fix stuck cursor on labels #1801 (@div72) - beacon: Fix research wizard beacon renewal status #1805 (@cyrossignol) - gui: Fix translations for port numbers #1818 (@cyrossignol) - util: Create parent directory #1821 (@div72) - mining: Fix coinstake/claim signature order #1828 (@cyrossignol) - voting: Remove double increment in loop #1831 (@cyrossignol) - neuralnet, scraper: Fix compilation with gcc5 and older libcurl #1832 (@a123b) - wallet: Fix smallest coin selection for contracts #1841 (@cyrossignol) - gui: Fix display of polls with no votes yet #1844 (@cyrossignol) - gui: add indentation to diagnostic status bar labels #1849 (@jamescowens) - voting, gui: Fix formatting and alignment of vote shares and percent #1850 (@jamescowens) - wallet, rpc: Fix for self-transactions in listtransactions #1852 (@jamescowens) - accrual: Clear any accrual snapshots when syncing from pre-v11 #1853 (@cyrossignol) - accrual: Fix reset of accrual directory if starting sync below research age height #1857 (@jamescowens) - gui: Fix researcher wizard layout on macOS with native theme #1860 (@cyrossignol)
Adds lint scripts and a Travis job for them, also includes some typo fixes.